Skip to content

Change host's memory setup to be minimally configured #297

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 26 commits into
base: main
Choose a base branch
from

Conversation

danbugs
Copy link
Contributor

@danbugs danbugs commented Feb 27, 2025

We want to allow embedding custom guests in Hyperlight, but, currently, the static memory layout we configure in the host is too strict. This PR is modifying our memory layout to be minimally configured.

Here's an example of how a guest could address its memory:
image

Changes:

  • BASE_ADDRESS modified to be 0x0 instead of 0x200_000 (2MB), removing the arbitrary unmapped section of memory.
  • Moved guest code section to be at 0x0, shifting paging sections up.
  • Added undifferentiated custom guest memory section that the guest can address in any way it wants.
  • Added API in common library to allow the guest to address the undifferentiated memory region.
  • Introduces SandboxBuilder API.
  • Removes MemMgrWrapper abstraction.

TODO:

  • fix tests/fuzzing/examples
  • make it work for all drivers,
  • remove uneeded entrypoint param,
  • add OutBAction::DebugPrint instead of HostPrint host fxn,
    • reconsider seccomp filter API now that we don't have the "HostPrint" default host function.
  • test Nanvix compat,
  • set right rsp,
  • reconsider host error and host fxn details memory sections, and
  • re-add crashdump feature.
  • fix mem access handler.

@danbugs danbugs added kind/refactor For PRs that restructure or remove code without adding new functionality. kind/enhancement For PRs adding features, improving functionality, docs, tests, etc. labels Feb 27, 2025
danbugs added a commit to danbugs/hyperlight that referenced this pull request Feb 27, 2025
…st code to start at 0x0

instead of after paging sections

This is the first step in moving memory sections around to provide a more generic
layout. After this, I'll move continue moving sections to fulfill the layout shown
in the description for PR hyperlight-dev#297.

Changes:
- removed no longer applicable checks in hypervisor_handler
- changed the way we build memory in get_memory_regions
- added functions to get the offsets/addresses to paging sections
as they are no longer static from 0x0 and are now relative to code size
- updated code docs
- changed set_up_shared_memory to properly place paging sections
- updated paging docs

Signed-off-by: danbugs <[email protected]>
danbugs added a commit to danbugs/hyperlight that referenced this pull request Feb 27, 2025
…st code to start at 0x0

instead of after paging sections

This is the first step in moving memory sections around to provide a more generic
layout. After this, I'll move continue moving sections to fulfill the layout shown
in the description for PR hyperlight-dev#297.

Changes:
- removed no longer applicable checks in hypervisor_handler
- changed the way we build memory in get_memory_regions
- added functions to get the offsets/addresses to paging sections
as they are no longer static from 0x0 and are now relative to code size
- updated code docs
- changed set_up_shared_memory to properly place paging sections
- updated paging docs

Signed-off-by: danbugs <[email protected]>
@danbugs danbugs force-pushed the danbugs/memory-layout branch from 56852a6 to 40a8091 Compare February 27, 2025 16:59
danbugs added a commit to danbugs/hyperlight that referenced this pull request Feb 28, 2025
…st code to start at 0x0

instead of after paging sections

This is the first step in moving memory sections around to provide a more generic
layout. After this, I'll move continue moving sections to fulfill the layout shown
in the description for PR hyperlight-dev#297.

Changes:
- removed no longer applicable checks in hypervisor_handler
- changed the way we build memory in get_memory_regions
- added functions to get the offsets/addresses to paging sections
as they are no longer static from 0x0 and are now relative to code size
- updated code docs
- changed set_up_shared_memory to properly place paging sections
- updated paging docs

Signed-off-by: danbugs <[email protected]>
@danbugs danbugs force-pushed the danbugs/memory-layout branch 3 times, most recently from b7435ae to ce255f8 Compare February 28, 2025 02:06
danbugs added a commit to danbugs/hyperlight that referenced this pull request Mar 17, 2025
…st code to start at 0x0

instead of after paging sections

This is the first step in moving memory sections around to provide a more generic
layout. After this, I'll move continue moving sections to fulfill the layout shown
in the description for PR hyperlight-dev#297.

Changes:
- removed no longer applicable checks in hypervisor_handler
- changed the way we build memory in get_memory_regions
- added functions to get the offsets/addresses to paging sections
as they are no longer static from 0x0 and are now relative to code size
- updated code docs
- changed set_up_shared_memory to properly place paging sections
- updated paging docs

Signed-off-by: danbugs <[email protected]>
@danbugs danbugs force-pushed the danbugs/memory-layout branch from 8aa7f89 to f21aabc Compare March 17, 2025 00:46
danbugs added a commit to danbugs/hyperlight that referenced this pull request Mar 17, 2025
…st code to start at 0x0

instead of after paging sections

This is the first step in moving memory sections around to provide a more generic
layout. After this, I'll move continue moving sections to fulfill the layout shown
in the description for PR hyperlight-dev#297.

Changes:
- removed no longer applicable checks in hypervisor_handler
- changed the way we build memory in get_memory_regions
- added functions to get the offsets/addresses to paging sections
as they are no longer static from 0x0 and are now relative to code size
- updated code docs
- changed set_up_shared_memory to properly place paging sections
- updated paging docs

Signed-off-by: danbugs <[email protected]>
@danbugs danbugs force-pushed the danbugs/memory-layout branch from f21aabc to 53069f2 Compare March 17, 2025 15:50
danbugs added a commit to danbugs/hyperlight that referenced this pull request Mar 19, 2025
In hyperlight-dev#297, we make the base address of guest memory 0x0 instead of 0x200_000 and
remove statically defined memory regions. This commit updates documentation in
accordance to these changes.

Signed-off-by: danbugs <[email protected]>
@danbugs danbugs force-pushed the danbugs/memory-layout branch 2 times, most recently from 4980140 to 3212be7 Compare March 19, 2025 05:31
danbugs added a commit to danbugs/hyperlight that referenced this pull request Mar 19, 2025
In hyperlight-dev#297, we make the base address of guest memory 0x0 instead of 0x200_000 and
remove statically defined memory regions. This commit updates documentation in
accordance to these changes.

Signed-off-by: danbugs <[email protected]>
@danbugs danbugs force-pushed the danbugs/memory-layout branch from 3212be7 to be061f0 Compare March 19, 2025 05:31
danbugs added a commit to danbugs/hyperlight that referenced this pull request Mar 19, 2025
In hyperlight-dev#297, we make the base address of guest memory 0x0 instead of 0x200_000 and
remove statically defined memory regions. This commit updates documentation in
accordance to these changes.

Signed-off-by: danbugs <[email protected]>
@danbugs danbugs force-pushed the danbugs/memory-layout branch from be061f0 to 76d67a8 Compare March 19, 2025 15:23
@danbugs danbugs removed the kind/refactor For PRs that restructure or remove code without adding new functionality. label Mar 19, 2025
danbugs added a commit to danbugs/hyperlight that referenced this pull request Apr 11, 2025
In hyperlight-dev#297, we make the base address of guest memory 0x0 instead of 0x200_000 and
remove statically defined memory regions. This commit updates documentation in
accordance to these changes.

Signed-off-by: danbugs <[email protected]>
@danbugs danbugs force-pushed the danbugs/memory-layout branch 3 times, most recently from 06dd5e3 to 6e82f64 Compare April 14, 2025 22:49
danbugs added a commit to danbugs/hyperlight that referenced this pull request Apr 14, 2025
In hyperlight-dev#297, we make the base address of guest memory 0x0 instead of 0x200_000 and
remove statically defined memory regions. This commit updates documentation in
accordance to these changes.

Signed-off-by: danbugs <[email protected]>
@danbugs danbugs force-pushed the danbugs/memory-layout branch 2 times, most recently from cbf1071 to 63e3c69 Compare April 14, 2025 23:03
danbugs added 21 commits April 21, 2025 16:56
In hyperlight-dev#297, we make the base address of guest memory 0x0 instead of 0x200_000 and
remove statically defined memory regions. This commit updates documentation in
accordance to these changes.

Signed-off-by: danbugs <[email protected]>
…yout + removes MemMgrWrapper

This commit completely removes the file w/ the mem layout. There are still
references to it around, but those will be removed in subsequent commits.

With the upcoming refactors, there is no need for the MemMgrWrapper abstraction. This
commit removes in in favour of directly utilizing a SandboxMemoryManager<S> in subsequent
commits.

Signed-off-by: danbugs <[email protected]>
- removes old HyperlightPEB structures (making it more idiomatic Rust).
- removes input/output structures from guest library, moving it to common as they
will need to be used by external guests.
- creates new HyperlightPEB API w/ capabilities to address memory regions from the guest.

Signed-off-by: danbugs <[email protected]>
…uilder

This commit partially introduces the SandboxBuilder mentioned on hyperlight-dev#249, changing
the way we create uninitialized sandboxes.

- adds SandboxBuilder API, which allows for a more flexible memory setup.
- changes to uninit w/ how it is created.
- deletes memory_region.rs file w/ most of it being superseded by the SandboxBuilder structures.
- simplifies config.rs removing all items related to previous static memory layout.

Signed-off-by: danbugs <[email protected]>
… in common library and use CGM

The host now defines a single undifferentiated memory region—the custom guest memory region (or, CGM)—that
the guest can address as it sees fit. This commit changes the guest library to leverage that and perform
said address using the APIs defined in the common library.

Signed-off-by: danbugs <[email protected]>
- removes static layout from SandboxMemoryManager, adds SandboxMemorySections structure built
w/ the SandboxBuilder.
- updates paging to reflect usage of dynamic memory layout.
- interact with common APIs.
- add APIs to allow reading addressed memory sections in PEB from shared memory.

Signed-off-by: danbugs <[email protected]>
… codebase

- temporarily comment out a lot of code (e.g., host/testing/mod.rs).
- temporarily unuse seccomp stuff.
- start using new mgr API and common API.
- pending removal of writting host detail functions to memory.
- change hv_handler to use new mgr API to get guest_function_dispatch_ptr.
- remove layout/memory_region from mod.rs.
- change RawPtr API (should probably be re-evaluated).

Signed-off-by: danbugs <[email protected]>
Updates drivers to map memory region in accordance to CGM usage.

> Note: only updated KVM fully so far. WHP and inprocess drivers are TODOs.

Signed-off-by: danbugs <[email protected]>
Signed-off-by: danbugs <[email protected]>
…d a DebugPrint OutBAction

- the mechanism for causing VM exits and its abstractions over host functions should be
exposed in the common library so they can be used externally.
- 'PrintOutput' should not be a special host function. Instead, we should provide functionality for
debug prints—i.e., the DebugPrint OutBAction.

Signed-off-by: danbugs <[email protected]>
If there's a mismatch in the guest's usize and the host's usize, our input and output
stacks would break. This commit fixes that by using u64 instead of usize for ptr accesses.

Signed-off-by: danbugs <[email protected]>
Removed an uneeded entrypoint argument for the size of the PEB structure, which was a re-use of the ops argument
in dev that is also unused.

+ cleaned up some TODOs (better error messages for unwraps and uneeded host fxn details section).

Signed-off-by: danbugs <[email protected]>
(1) had to add helper impl functions for the HyperlightPEB struct to get the host addresses for some memory regions.
(2) re-added some logic for the LeakedOutBWrapper and for the InProcessDriver.
(3) added an in process test in the sandbox_builder file.
(4) renamed hyperlight_common::hyperlight_peb mod to hyperlight_common::peb.

Signed-off-by: danbugs <[email protected]>
Needed to make some modications to hyperv_linux for it to conform to the new SandboxMemorySections API. With that,
I also had to add a From impl to sandbox_builder. As an aside, I also modified the signature of get_memory_access_violation
to take a borrow of SandboxMemorySections instead of an owned value, which ended up impacting kvm too.

Signed-off-by: danbugs <[email protected]>
Changed whp driver to use the new SandboxMemorySections API.

Signed-off-by: danbugs <[email protected]>
…shdump feature

Crashdump feature used the old MemoryRegion API. Updated it to use the new SandboxMemorySections API (causing
changes in all drivers) and added a test in sandbox_builder for the feature.

Signed-off-by: danbugs <[email protected]>
- cleaned up the HyperlightPEB API in the common library.
-- added MemoryRegion struct to better group related offsets and sizes.
-- removed pub fields from HyperlightPEB struct making fields accessible only via getters/setters.
-- cleaned up, commented, and re-organized existing fxns for HyperlightPEB struct.
- now we modify the rsp in the host if the guest sets up a new stack region (i.e., essentially dropping
the tmp stack).

Signed-off-by: danbugs <[email protected]>
- the guest error data region was removed and now, instead, we leverage the output data region to communicate guest errors back to the host.
- previously, the host error data region was incorrectly accessed making it essentially unused. This commit removes the region altogether. There is
some merit to the region existing in some way, but this is outside of the scope of hyperlight-dev#297.

Signed-off-by: danbugs <[email protected]>
- updated our hello-world example,
- updated the README,
- changed last remaining use of HostPrint to use print (wrapper for using the DebugPrint OutbAction), and
- added an extra test in sandbox_builder.

Signed-off-by: danbugs <[email protected]>
…dler + re-added seccomp test

Previously set mem_access_handler_fn to be NOOP. Brought back original functionality;
albeit a bit questionable.

Signed-off-by: danbugs <[email protected]>
@danbugs danbugs force-pushed the danbugs/memory-layout branch 2 times, most recently from d126ecc to 846780a Compare April 21, 2025 17:36
danbugs added 2 commits April 21, 2025 19:33
Tests are already covered in the sandbox_builder. In 75b0076, I also added
a test for Windows in-process mode using the load library.

Signed-off-by: danbugs <[email protected]>
- Changed the ABI for the inprocess outb handler from `extern "C"` to
`extern sysv64`. This was needed because `extern "C"` caused issues on Windows.
Previously, we used `extern "win64"`, which was problematic for guests
intending to use `hyperlight_common` and not supporting the Windows ABI.
- Added a test for running inprocess on Windows w/ load library.

Signed-off-by: danbugs <[email protected]>
@danbugs danbugs force-pushed the danbugs/memory-layout branch from 75b0076 to 04e8e96 Compare April 21, 2025 19:34
Signed-off-by: danbugs <[email protected]>
@danbugs danbugs force-pushed the danbugs/memory-layout branch from 3c6255b to 815cb5b Compare April 21, 2025 23:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind/enhancement For PRs adding features, improving functionality, docs, tests, etc.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant